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Introduction 


Our  research  has  been  focused  on  three  goals. 

The  first  goal  was  to  find  ways  of  describing  various  thin- 
nings of  point  processes  that  arise  for  example  in  the 
debugging  of  computer  programs.  Our  work  on  this  lead  us 
to  studies  of  extreme  values  of  stochastic  processes  and 
the  convergence  of  intergrals.  Our  results  in  these  areas 
are  described  in  Sections  I-III. 

The  second  goal  was  to  determine  optimal  policies 
for  dynamically  controlling  certain  queueing  processes.  In 
order  to  do  this,  we  had  to  answer  some  basic  questions  in 
dynamic  programming  concerning  monotone  optimal  policies 
and  the  equivalence  of  continuous  and  discrete  time  models. 
We  discuss  this  in  Section  IV. 

Our  third  goal,  as  we  describe  in  Section  V,  was 
to  continue  developing  the  theory  and  applications  of  semi- 
stationary  processes.  The  emphasis  here  was  on  semi- 
stationary  clearing  processes  which  are  useful  for  modeling 
input-output  systems  such  as  batch  service  queues  and 
Inventory  systems. 
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I . Thlnnlnris  of  Point,  Procei^ses  and  Random  Measuros 

Most  of  our  research  duriti;^  the  last  ^ years  has 
been  on  thinnings  of  random  measures.  A random  measure 
refers  to  randomly  located  masses  or  points  In  a region, 
and  thinning  of  It  means  tliat  some  of  this  mass  Is  randomly 
deleted.  A point  process  Is  a special  random  measure  having 
only  unit  masses.  Such  thinnings  arise  In  debugging  com- 
puter software  packages,  filtering  particles  from  a solution, 
deleting  bad  data,  and  the  wearing  of  physical  surfaces  such 
as  ball  bearings  or  tires.  Over  the  last  two  decades  about 
twenty  papers  have  been  written  on  thinnings  of  point 
processes.  These  studies  focused  on  the  "locations  of 
points"  of  the  processes  and  consequently  they  appear  to  be 
very  different  mathematically.  We  took  a different  approach 
In  [1]  and  described  thinnings  by  looking  at  the  number  of 
points  deposited  and  thinned  In  various  "regions".  In  doing 
so,  we  found  that  essentially  all  of  the  thinnings  studied 
to  date  were  compositions  of  random  measures.  This  Idea 
enabled  us  to  unify  and  significantly  extend  this  newly 
developing  theory  of  thinning. 

Our  results  in  [1]  are  basically  convergence 
theorems  for  various  compositions  of  measures.  These 
answer  questions  such  as:  (1)  What  Is  the  end  result  when 
one  applies  a particular  thinning  to  a random  measure? 
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(2)  When  Is  the  end  result  a Poisson,  Compound  Poisson  or 
Infinitely  divisible  random  measure  and  what  are  their 
defining  parameters?  (3)  How  does  the  randomness  of  the 
initial  masses  interact  with  the  randomness  of  the  thinning 
operation?  (4)  What  does  it  mean  to  thin  in  a continuous 
manner  such  as  the  wearing  or  erosion  of  physical  objects 
or  land? 

In  analyzing  thinnings  we  found  that  inverses  of 
random  measures  played  a key  role.  We  proved  some  basic 
properties  of  such  Inverses  which  are  also  useful  in  other 
contexts . 

Our  second  thinning  paper  [2]  is  a study  of 
interchangeable  thinnings.  This  means  that  the  probabilistic 
nature  of  the  thinning  is  invariant  under  rearrangement  of 
the  masses.  This  is  more  natural  that  independent  identi- 
cally distributed  thinnings  for  many  applications.  We  study 
botn  discrete  and  continuous  thinnings.  Here  we  view  thin- 
nings as  special  cases  of  compounding  where  mass  can  be 
Increased  or  decreased.  Several  results  that  unify  and 
extend  earlier  results  by  Renyl  and  Kallenberg  on  inde- 
pendent thinnings  are  proved.  This  paper  contains  several 
results  we  documented  earlier  on  thinnings  of  point  processes 
over  time.  In  these  contexts  thinning  Is  done  over  a long 
time  period  and  the  resulting  point  processes  is  analyzed 
by  looking  at  the  lifetime  of  the  points  under  the  thinning. 
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If  the  lifetime  distributions  have  certain  types  of  tails, 
then  the  thinned  process  after  a long  time  converges  to  a 
Poisson  or  infinitely  divisible  point  process  or  such  a 
process  with  randomized  parameters.  One  may  also  be  in- 
terested in  the  residual  lives  of  the  points  after  a long 
period.  This  tells  how  much  more  thinning  time  is  needed 
to  completely  thin  the  points.  We  also  show  how  to  analyze 
such  residual  random  measures. 

The  last  part  of  [2]  is  a study  of  sequential 
thinnings  of  points  on  the  real  line.  Here  a single  thin-  ! 

nlng  operation  over  the  whole  line  deletes  points  by  a rule 
such  that  the  number  of  points  retained  between  deleted  ^ 

points  are  independent  and  Identically  distributed.  A | 

sequence  of  these  thinning  operations  are  performed  and  the 
thinned  process  tends  to  be  a renewal  process  whose  inter- 
point distribution  is  a certain  limiting  distribution  that 
arises  in  the  theory  of  branching  processes.  Mathematically 
this  analysis  is  a study  of  the  convergence  of  a large 
number  of  compositions  of  random  measures  as  in  [1].  This 
is  of  Interest  by  Itself:  it  may  be  of  use  in  other 
contexts . 

Just  recently  we  started  analyzing  thinnings  of 
cluster  point  processes.  Namely  cluster  centers  are  randomly 
placed  in  a region  and  each  center  gives  rise  to  a point 
process.  The  total  system  of  points  is  a cluster  point 
process.  A typical  thinning  may  delete  a cluster  center, 
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and  hence  its  related  point  process.  Also  points  may  be 
deleted  Individually.  It  appears  that  our  various 
approaches  for  analyzing  thinnings  can  handle  these  more 
complex  systems.  We  will  document  this  next  year. 
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II . Extreme  Values  of  Stochastic  Processes 

Some  problems  concerning  extreme  values  of 
stochastic  processes  are  actually  thinning  problems.  For 
example,  consider  a discrete  time  stochastic  process  and 
let  be  the  number  of  times  that  this  process  exceeds 

the  value  n In  time  t.  As  n Increases  the  process 
becomes  thinner  (contains  fewer  points).  Similarly  let 
Mj^(t)  be  the  total  amount  of  these  exceedances  above  the 
level  n.  This  Is  a random  measure  which  also  becomes 

thinner  as  n Increases. 

We  study  the  thinning  of  such  exceedance  measures 
in  [3]  when  the  basic  stochastic  process  is  a recurrent 
Markov  chain,  a regenerative  process  or  a semi-stationary 
process.  We  present  necessary  and  sufficient  conditions 
for  the  convergence  of  random  measures  related  to  exceedances. 

In  particular  we  show  that  high  level  exceedances 
of  certain  random  walks  and  queueing  processes  are  compound 
Poisson  processes.  This  extends  some  earlier  work  by 
Llndvall.  The  results  here  are  also  useful  for  describing 
thinnings  of  point  processes  over  time  where  the  initial 
points  or  the  thinning  mechanism  has  certain  Markovian  or 
regenerative  dependencies. 

One  problem  in  thinning  over  time  is  to  find  out 
how  long  it  may  take  to  eliminate  all,  or  maybe  90  percent, 
of  the  points.  This  can  be  described  by  the  convergence  of 
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the  maximum  or  k-th  maximum  of  a large  number  of  random 
variables.  Namely,  the  variables  are  the  lifetimes  of  the 
points  under  the  thinning  operation.  During  the  last  year 
we  have  obtained  several  results  on  extremes  of  Independent 
and  dependent  variables.  One  significant  result  is  a 
necessary  and  sufficient  condition  for  extreme  values  of 
Independent  infinitesimal  processes  to  converge.  This  is 
analogous  to  the  general  functional  central  limit  theorem 
for  sums  of  independent  infinitesimal  variables.  We  are 
still  studying  the  many  ramifications  of  this  result  and 
its  extension  to  multidimensional  settings  where  there  are 
many  applications.  We  hope  to  document  our  results  in  this 
area  next  year. 
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III . Convergence  of  Intep:rals 

Our  study  of  thinnings  lead  us  to  study  the  con- 
vergence in  distribution  of  random  Integrals  of  the  form 

^ Y = f Y (t)dC  (t) , 

^n  n J ^ n n ’ 

where  ^ is  a random  measure  on  a space  X and  Y is  a 
random  function,  and  the  has  certain  asymptotic 

properties.  The  in  our  contexts  was  usually  an  infinite 
random  measure  (e.g.,  a Poisson  process),  and  so  these 
integrals  could  not  be  analyzed  by  the  standard  theory  for 
finite  measures. 

In  [^]  we  study  the  convergence  of  nonrandom  and 
random  Integrals.  The  basic  results  are  necessary  and 
sufficient  conditions  for  the  convergence  of  nonrandom 
intergral  p f = f (x)dp  (x)  where  p and  f converge  in 
some  sense.  These  are  analogous  to  the  classical  result 
that  if  nonnegative  f^  converge  to  f in  p-measure  then 
|f^dp  -*■  jfdp  if  and  only  if  the  are  uniformly 
p-lntegrabla  We  present  similar  results  for  the  convergence 
of  measures  v defined  by 

V (AxB)  = I k (x,B)dp  (x)  for  AxBCXxY 
n n n 

where  p^  is  a measure  on  X and  k is  a kernel  from  X to  Y. 
n n 

We  then  show  how  these  results  apply  to  random  integrals 
and  measures.  A major  application  pertains  to  the  convergence 
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of  mixtures  of  probabilities  that  is  a key  to  describing 
extremes  of  exchangeable  random  variables. 


10 


IV.  optimal  Control  of  Random  Vfalko,  Birth  and  Death 
Processes  and  Queues 

The  first  result  In  queueing  control  appeared  a 
decade  ago.  Since  then  there  have  been  many  ad  hoc  studies 
of  queueing  control  problems,  but  a nice  unified  theory  has 
not  yet  been  developed.  Two  reasons  for  this  are  (1)  the 
state  space  of  these  models  is  often  infinite,  and  (2)  a 
slight  change  in  the  cost  function  of  a model  may  result 
in  a drastic  change  in  its  mathematical  analysis.  One  of 
our  research  goals  was  to  solve  some  basic  queueing  control 
problems  and  to  unify  part  of  the  theory  dealing  v/ith 
monotone  optimal  policies. 

The  major  problem  we  address  in  [7]  is  a follows. 
Consider  a single  sewer  queueing  system  with  Poisson 
arrivals  and  exponential  service  times  (an  M/M/1  queue) 
whose  arrival  and  service  rates  are  controllable.  At  each 
customer  arrival  or  departure  the  number  of  customers  in 
the  system  is  observed.  Based  on  this  number  an  arrival 
and  service  rate  are  selected  from  a prescribed  set.  Costs 
are  charged  for  using  these  rates  and  for  holding  customers 
in  the  system.  The  aim  is  to  find  a policy  for  successively 
selecting  the  arrival  and  service  rates  so  as  to  minimize 
the  expected  discounted  or  average  cost  of  the  system. 

Under  some  natural  conditions  on  the  costs  there 
should  be  an  optimal  policy  which  increases  the  service 
rate  and  decreases  the  arrival  rate  as  the  number  of  waiting 
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customers  increases.  The  following'  questions  arise: 

(1)  What  are  natural  conditions  on  the  costs 
that  lead  to  such  monotone  optimal  policies? 

(2)  How  can  one  prove  the  existence  of  a 
monotone  optimal  policy? 

(3)  Knowing  that  a monotone  optimal  policy 
exists,  how  can  it  be  computed  especially 
for  an  infinite  or  large  state  space? 

We  answer  these  questions  in  [?]• 

A major  theme  in  queueing  and  inventory  control 

/ 

problems  is  to  establish  the  existence  of  monotone  optimal 
policies.  This  reduces  the  search  for  an  optimal  policy 
to  the  subclass  of  monotone  policies,  and  the  computation 
of  these  policies  becomes  easier.  In  [5]  we  present 
general  conditions  under  which  Markov  decision  processes 
have  monotone  optimal  policies.  We  Illustrate  this  for  a 
random  walk  problem  and  for  a multimachine  replacement 
problem.  Some  of  the  techniques  in  [7]  are  also  extended 
and  used  in  our  queueing  analysis  in  [5]-  We  found  that 
essentially  all  of  the  queueing  and  Inventory  control 
models  that  have  been  studied  are  controlled  random  walks 
or  Markov  chains  with  monotone  matrices.  That  is  why  they 
have  monotone  optimal  policies.  These  are  a very  small 
class  of  decision  processes.  It  seems  like  other  sub- 
classes of  decision  processes  should  also  have  monotone 
optimal  policies,  but  we  have  not  been  able  to  find  con- 
ditions to  identify  such  subclasses. 
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A key  step  in  our  queueing  control  paper  [5]  was 
to  show  that  a continuous  time  queueing  process  is  equivalent 
to  a discrete  time  random  walk.  This  enabled  us  to  analyze 
an  ostensibly  more  complex  continuous  time  process  in  terms 
of  a simpler  discrete  time  process.  Such  an  equivalence  also 
holds  for  general  continuous  and  discrete  time  processes  as 
we  discuss  in  [6].  This  study  clarifies  some  earlier  work 
dn  the  equivalence. 
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V.  Seml-Stat lonary  Clearlnr^  Processes 

Many  input-output  systems  such  as  batch  services 
queues,  dams,  inventories,  and  dispatching  of  men  and 
equipment  can  be  viewed  as  a clearing  process.  Such  a 
process  describes  the  net  quantity  in  a system  which 
receives  an  exogeneous  random  input  over  time  and  has  an 
output  mechanism  that  intermittently  clears  random  quanti- 
ties from  the  system.  A semistationary  clearing  process 
is  strictly  stationary  over  its  clearing  epochs. 

We  study  these  processes  in  [8].  We  describe 
their  asymptotic  distribution  and  show  how  it  arises  in 
the  limit  of  certain  functionals.  An  asymptotic  distribu- 
tion is  different  from  a limiting  distribution,  but  it  has 
some  similiar  properties.  We  identify  clearing  processes 
whose  asymptotic  distribution  is  uniform.  One  example  is 
a modulo  c clearing  process  in  which  an  amount  c is  removed 
whenever  that  amount  is  exceeded.  For  this  example  the 
Palm  probability  of  the  process  must  be  used  rather  than 
the  usual  probability  to  obtain  a uniform  distribution. 

This  subtle  use  of  the  Palm  probability  gives  a partial 
answer  to  an  anomaly  in  a classical  economic  lot  size 
inventory  model.  We  also  present  several  limit  theorems 
for  clearing  processes  including  a functional  central  limit 
law  and  law  of  the  Iterated  algorithm. 
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20.  Abstract  continued. 


'eind  the  equivalence  of  continuous  and  discrete  time  models. 

The  third  goal,  was  to  continue  developing  the  theory  applications  of 
semi— stationary  processes.  The  emphasis  here  was  on  semi— stationary 
clearing  processes  which  are  useful  for  modeling  input-output  systems 
such  as  batch  service  queuts  and  Inventory  systems. 
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